Recurrent neural network-enhanced HMM speech recognition systems

Keywor ds : speech segmentation, speech recognition, recurrent neural networks, extended recurrent neural networks, bi-directional recurrent neural networks, hidden Markov models Speech recognition is fast becoming an attractive form of communicat ion between humans and machines, due to the recent advances made in the field. The technology is, however, far from being acceptable when used in an unlimited form. Many attempts have been made to overcome some of the many difficu lties faced in the automatic recognition of human speech by machine. This dissertation attempts to accomplish two ambi tious goals. The first is t he reliable, automatic segmentation of speech signals, in a speaker independent manner, where no higher level lexical knowledge is used in the process. The system is limited to segmentation in an off-line manner. T he second is to improve the phoneme recognit ion accuracy of a state-ofthe-art hidden Markov model (HMM) based recognition system, using the segmentation information. A new technique of incorporating segmentation information into the Viterbi decoding process is presented, and it is shown that this technique outperforms other attempts to include the segmentation probabilities. T he segmentat ion system consists of a bi-directional recurrent neural network (BRNN), also called an extended recurrent neural network (ERNN). In contrast to convent ional recurrent neural networks, that only use speech vectors from the past, present and possibly a fixed window of the future, BRNNs use all of t he past, present and future